The present invention relates to autonomous vehicle systems and, more particularly, to devices and systems for enabling autonomous vehicles to communicate with legacy vehicles, to allow legacy vehicles to receive and respond to informational content from other autonomous vehicle data stream messaging and the Intelligent Transportation System (ITS).
An autonomous car or other autonomous vehicle (also known as a driverless car, auto, self-driving car, robotic car) is a vehicle that is capable of sensing its environment and navigating without human input. Many such vehicles are being developed, but currently few, if any, automated cars permitted on public roads are fully autonomous, and may require a human driver at the wheel who is ready at a moment's notice to take control of the vehicle.
Autonomous cars use a variety of techniques to detect their surroundings, such as radar, laser light, GPS, odometry, and computer vision. Advanced control systems interpret sensory information to identify appropriate navigation paths, as well as obstacles and relevant signage. Autonomous cars have control systems that are capable of analyzing sensory data to distinguish between different cars on the road, which is very useful in planning a path to the desired destination.
“Autonomous” generally means having the power for self-governance. Autonomous control implies good performance under significant uncertainties in the environment for extended periods of time and the ability to compensate for system failures without external intervention.
It is often suggested to enhance the capabilities of an autonomous car by implementing communication networks both in the immediate vicinity (for collision avoidance) and far away (for congestion management). By bringing in these outside influences in the decision process, some would no longer regard the car's behavior or capabilities as “autonomous”. In some context, “automated” connotes control or operation by a machine, while ‘autonomous’ connotes acting alone or independently. For purposes, of the present invention, as described below, the terms “autonomous” or “self-driving” are interpreted consistently, and to include “automated” vehicle systems. However, the latter term is “autonomous” is still considered accurate. Also, the terms “cars”, “vehicles” and “automobiles” are intended to include any transportation industry vehicle, using any power system, e.g., gasoline, electric, compressed natural gas, fuel cell, etc.
Modern self-driving cars generally use Bayesian Simultaneous Localization and Mapping (SLAM) algorithms, which fuse data from multiple sensors and an off-line map into current location estimates and map updates. SLAM with detection and tracking of other moving objects (DATMO), which also handles things such as cars and pedestrians, is a variant developed by research at Google Inc. Simpler systems may use roadside real-time locating system (RTLS) beacon systems to aid localisation. Typical car sensors include LIDAR, stereo vision, GPS and inertial measuring unit (IMU) sensors. Visual object recognition uses machine vision including neural networks. Educator Udacity is understood to have developed an open-source software stack.
Among the anticipated benefits of autonomous cars, and the intelligent transportation system (ITS) in which they participate, is the potential reduction in traffic collisions (and resulting deaths and injuries and costs), caused by human-driver errors, such as delayed reaction time, tailgating, rubbernecking, and other forms of distracted or aggressive driving. Consulting firm McKinsey & Company, Inc. is reported to have estimated that widespread use of autonomous vehicles could “eliminate 90% of all auto accidents in the United States, prevent up to US $190 billion in damages and health-costs annually and save thousands of lives”.
Autonomous cars are also predicted to offer major increases in traffic flow; enhance mobility for children, the elderly, disabled and poor people; lower fuel consumption; reduce the need for insurance; reduce the need for parking space in cities; a reduce vehicle associated crime; and the facilitate different business models for mobility as a service, especially for those involved in the sharing economy.
If a human driver isn't required, autonomous vehicles could also reduce labor costs; relieve travelers from driving and navigation chores, thereby replacing behind-the-wheel commuting hours with more time for leisure or work; and also would lift constraints relating to an occupants' ability to drive. As such there would be fewer incidents of drivers being distracted or texting while driving, intoxicated, prone to seizures, or otherwise impaired. For the young, the elderly, people with disabilities, and low-income citizens, autonomous vehicles could provide enhanced mobility.
Additional reported advantages associated with autonomous vehicle, could include higher speed limits; smoother rides; and increased roadway capacity; and reduced traffic congestion, due to decreased need for safety gaps and higher speeds. For example, currently, maximum controlled-access highway throughput or capacity according to the U.S. Highway Capacity Manual is reported to be about 2,200 passenger vehicles per hour per lane, with about 5% of the available road space is taken up by cars. According to a study by researchers at Columbia University, autonomous cars could increase capacity by 273% (approximately 8,200 cars per hour per lane). The study also estimated that with 100% connected vehicles using vehicle-to-vehicle communication, capacity could reach 12,000 passenger vehicles per hour (up 445% from 2,200 passenger vehicles/lane/hour) traveling safely at 120 km/h (75 mph) with a following gap of about 6 m (20 ft.) of each other. Currently, at highway speeds drivers keep between 40 to 50 m (130 to 160 ft.) away from the car in front. These increases in highway capacity could have a significant impact in traffic congestion, particularly in urban areas, and even effectively end highway congestion in some places.
As such, autonomous vehicles are expected to provide an improved ability to manage traffic flow, combined with less need for traffic police, less vehicle insurance, and even less road signage, since automated cars could receive necessary communication electronically (although roadway signage would still be needed for any human drivers on the road). Further, reduced traffic congestion and the improvements in traffic flow due to widespread use of autonomous vehicles are expected to also translate into better fuel efficiency.
Widespread adoption of autonomous cars could also reduce the needs of road and parking space in urban areas, freeing scarce land for other uses such as parks, public spaces, retail outlets, housing, and other social uses. Some believe that autonomous vehicles could also contribute, along with automated mass transit, to make dense cities much more efficient and livable.
The increased awareness of autonomous vehicles could reduce car theft, while the removal of the steering wheel—along with the remaining driver interface and the requirement for any occupant to assume a forward-facing position—could give the interior of the cabin greater ergonomic flexibility. Large vehicles, such as motorhomes, could attain appreciably enhanced ease of use.
When used for car sharing, the total number of cars on the roads likely to be further reduced. Furthermore, new business models (such as mobility as a service) may develop, which aim to be cheaper than car ownership by removing the cost of the driver. Finally, the robotic car could drive unoccupied to wherever it is required, such as to pick up passengers or to go in for maintenance (eliminating redundant passengers).
Individual autonomous vehicles may benefit from information obtained from not only their own information system, but also from information systems with other vehicles in the vicinity, especially information relating to traffic congestion and safety hazards. Vehicular communication systems may use other vehicles and roadside units as the communicating nodes in a peer-to-peer network, providing each other with information. As a cooperative approach, vehicular communication systems can allow all cooperating vehicles to be more effective. According to a 2010 study by the National Highway Traffic Safety Administration, vehicular communication systems could help avoid up to 79 percent of all traffic accidents. Among connected cars, an unconnected one may be the weakest link and may be increasingly banned from busy high-speed roads.
The communications systems to implement the connected vehicle applications referred to above include vehicle-to vehicle (V2V) and vehicle-to-infrastructure (V2I) applications that require a minimum of one entity to send information to another entity. Broadly, short range communications that occur between a vehicle and any similarly equipped external object may be collectively referred to as “V2X” communications. For example, many vehicle-to-vehicle safety applications can be executed on one vehicle by simply receiving broadcast messages from one or more neighboring vehicles. These messages are not necessarily directed to any specific vehicle, but are meant to be shared with a vehicle population to support the safety application. In these types of applications where collision avoidance is desirable, as two or more vehicles talk to one another in a setting where a collision becomes probable, the vehicle systems can warn the vehicle drivers, or possibly take action for the driver, such as applying the brakes. Likewise, roadway infrastructure components, such as traffic control units, can observe the information broadcasts or otherwise sense vehicle traffic and provide a driver warning if there is a detected hazard (e.g., if a vehicle is approaching a curve at an unsafe speed or there is a crossing vehicle that is violating a red traffic signal phase).
Since V2X communication is a cooperative technology, the system is dependent on other similarly equipped entities to provide safety benefits. As such, V2X systems are subject to the network effect, where the value of the system increases as the fleet penetration increases. In the early years of deployment, certain safety and other features may only be available in a limited fashion, as number of communication vehicles is not sufficient to provide safety benefits on a large scale. Existing vehicles without communications equipment will not be able to communicate with newer vehicles that have been deployed with a V2X communications system. Therefore, it may be desirable to provide an aftermarket device that is able to be used with an existing vehicle to allow that vehicle to be capable of providing vehicle location and state information to other vehicles and enable a variety of V2X features on the host vehicle using location and state (of the road) information that is obtained from other communicating vehicles.
Connectability between autonomous vehicles includes a capability known as dedicated short range communications (DSRC) is a two-way short-to-medium-range wireless communications capability that permits very high data transmission critical in communications-based active safety applications. In Report and Order FCC-03-324, the Federal Communications Commission (FCC) allocated 75 MHz of spectrum in the 5.9 GHz band for V2X data streams used by Intelligent Transportations Systems (ITS) vehicle safety and mobility applications.
DSRC based communications is reportedly a major research priority of the Joint Program Office (ITS JPO) at the U.S. Department of Transportation (U.S. DOT) Research and Innovative Technology Administration (RITA).
DSRC was initially developed with a primary goal of enabling technologies that support safety applications and communication between vehicle-based devices and infrastructure to reduce collisions. DSRC is reportedly the only short-range wireless alternative currently available that provides designated licensed bandwidth for secure, reliable communications to take place. As noted above, that bandwidth is primarily allocated for vehicle safety and mobility applications by FCC Report and Order FCC 03-324; including fast network acquisition, as active safety applications require the immediate establishment of communication and frequent updates. DSRC also provides low latency, which is useful as active safety applications must recognize each other and transmit messages to each other in milliseconds without delay. DSRC further provides high reliability when required active safety applications require a high level of link reliability. DSRC works in high vehicle speed mobility conditions and delivers performance immune to extreme weather conditions (e.g. rain, fog, snow, etc.). DSRC provides priority for safety applications, in that safety applications on DSRC are given priority over non-safety applications; to support both V2V and V2I data streams; and safety message authentication and privacy.
Connected vehicle applications utilizing DSRC may have the potential to significantly reduce many of the most deadly types of crashes through real time advisories alerting drivers to imminent hazards—such as veering close to the edge of the road; vehicles suddenly stopped ahead; collision paths during merging; the presence of nearby communications devices and vehicles; sharp curves or slippery patches of roadway ahead.
Convenience services like e-parking and toll payment are also able to communicate using DSRC. Anonymous information from electronic sensors in vehicles and other devices, such as cellphones, and dongles can also be transmitted over DSRC to provide better traffic and travel condition information to travelers and transportation managers.
While the technology for connected vehicle applications has evolved considerably since the inception of autonomous vehicles, some related areas have lagged the development of suitable sensors and integration of sensor data with autonomous car applications. One such lagging area has been the development of devices that can be utilized to upgrade legacy vehicles (that do not include autonomous driving capabilities) to interface autonomous vehicle data streams. More particularly, there is an absence of portable devices that can be readily applied to legacy vehicles, without requiring any changes or modifications to the vehicle, which would permit legacy vehicles to be a part of the connected vehicle network.
Another area of shortcoming has been the sparse integration of vehicle diagnostic functions in devices utilized to interface with a connected vehicle data stream. While there have been disclosures suggesting the use of a vehicle diagnostic port in association with a connected vehicle application, such use is been limited, e.g., to implement diagnostic functions associated with the identification of terrain that would be suitable to engage in an analysis of mileage related tests or the like. However, there appears to have been little discussion respecting the combination of more encompassing diagnostic functions in association with devices used to interface vehicles with a connected vehicle data stream.
Diagnostic functions, may include identification of impending diagnostic conditions, e.g., low battery, low oil, low fuel, overheating, steering angle, vehicle identification number (VIN), BSMs, exterior light status, mileage, brake pad conditions, transmission gears, brake pedal status, ABS/SRS status, battery condition, TPMS tire pressure, seatbelt status of driver/passenger, turn signal status, data logger, ignition disable, fuel level, oil level, fuel level, emission system degradation, brake system failures and other information diagnostic functions are not only pertinent to the operation of the particular vehicle, but have bearing on how a particular vehicle interacts with other connected vehicles. For example, where a vehicle diagnostic system detects that a vehicle has a brake system condition, and communicates such information to the connected vehicle data stream, the speed and spacing of adjacent vehicle may be adjusted to accommodate that condition, with an appropriate safety margin. Alternatively, where diagnostic system determines that certain diagnostic sensor functions are inoperative, that information may also result in changes in the manner in which the particular vehicle is directed.
Ideally a device and system could be provided that incorporates enhanced diagnostic capabilities that could generate information that could be merged with a connected vehicle data stream, or utilized independent of the connected vehicle data stream, e.g., to diagnose and alert the driver of the diagnostic condition of the vehicle. As enhanced diagnostic capabilities are, in many respects tailored to the specific functional characteristics of a particular vehicle and associated parts, such enhanced diagnostic capabilities, may require access to a configuration database and a processor able to autonomously process the diagnostic information for a particular vehicle with a sufficiently low latency period to be suitable for use in conjunction with many connected vehicle applications.
It would further be desirable if such a device and system could be provided in an aftermarket product that is compatible with a wide range of legacy vehicles, can be easily installed by a vehicle owner and can be easily configured automatically based on the vehicle VIN at the time of installation.
U.S. Pat. No. 8,930,041 issued Jan. 6, 2015, to Grimm, et al., the contents of which are herein incorporated by reference, discloses an aftermarket plug-in safety device that allows a vehicle to communicate with other vehicles or infrastructures in a V2X communications system. The safety device is generally operable to be coupled to an OBD connector on the vehicle, and includes processing capabilities to identify the vehicle that it is coupled to by receiving data on a vehicle CAN bus, where the device performs self-configuring operations based on the type of vehicle, access to vehicle systems and location of the vehicle. The device also includes a radio for transmitting and receiving signals and a global navigation satellite system (GNSS) receiver for receiving location signals and providing vehicle position data. In this matter, the vehicle is able to communicate with other vehicles having similar communications capabilities.
U.S. Patent Publication No. 2017/0048080, to Grimm et al., the entire contents of which are herein also incorporated by reference, discloses an aftermarket communications device that can also be used in association with a vehicle to communicate with other vehicles or infrastructures in a V2X communications system. The communication device can be a plug-in device, a wireless device separate from the vehicle, such as a key fob, a smart phone, or a permanent retrofit device mounted to the vehicle. The communications device is electrically coupled to the vehicle by, for example, an OBD connection, a USB connection, a CAN bus connection, wireless connection or an HDMI connection. The communication device includes a radio for transmitting and/or receiving communications signals, a memory for storing security information and vehicle application data, a location processor such as a global navigation satellite system receiver and a verification processor configured to be put in electrical communication with a CAN bus on the vehicle, where the communications processor receives vehicle location signals from the location processor, files from the memory and signals from the radio.
Despite the advances in the development of connected vehicles, and proposals for devices to interface a connected vehicle data stream with existing “legacy” vehicles, there remains a need for a device that can readily be engaged to a vehicle, without the need for any modification of a vehicle or permanent attachment to a vehicle. Preferably the device is configured to not only interface the V2X data stream, but can also function to independently sense conditions, such as safety hazards in the vehicle environment, that can be used to actuate safety systems in the vehicle, and/or be communicated to other vehicles via the V2X data stream to allow actuation of safety systems in other vehicles as well. Such a device would be useful to allow many types of legacy vehicles to not only receive safety information from a V2X data stream, but also allows the legacy vehicle to display and act on the received information, and to contribute sensed information to the data stream, for use by other connected vehicles.
Preferably such a device would also incorporate a communication system that will allow the device to access enhanced vehicle diagnostic information, and to communicate the information to remote resources for autonomous evaluation. Such a communications system may include a handheld wireless communication circuit, in wireless communication with the device for displaying and processing the vehicle diagnostic information, and/or further communication of diagnostic information to remote database for evaluation. Alternatively, the device may include a cellular network communications circuit, to allow the device to communicate directly with a remote database or other resources, rather than communicate through the use of an intermediate wireless communication device.
By incorporating the various features described above, such a device would not only provide a means for generally interfacing legacy vehicles to a V2X data stream, but also derive information from the V2X data stream that can be used to regulate the operation of interface various vehicle display/actuating/diagnostic systems. Such a device would also facilitate the use of remote resources for processing vehicle diagnostic information and/or the V2X data stream in a manner suitable for cooperation with devices and systems of a particular legacy vehicle. By incorporating such connectivity and functionality within a single portable device, the resources of the vehicle, the V2X data stream, a handheld wireless communication device and/or a remote database may be collectively utilized to provide enhanced functionality in an efficiently distributed autonomous system.